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Abstract 

A mixed linear quadratic (MLQ, for short) optimal control problem is considered. The controlled 
stochastic system consists of two diffusion processes which are in different time horizons. There are 
two control actions: a standard control action u(-) enters the drift and diffusion coefficients of both 
state equations, and a stopping time r, a possible later time after the first part of the state starts, 
at which the second part of the state is initialized with initial condition depending on the first state. 
A motivation of MLQ problem from a two-stage project management is presented. It turns out that 
solving MLQ problem is equivalent to sequentially solve a random-duration linear quadratic (RLQ, 
for short) problem and an optimal time (OT, for short) problem associated with Riccati equations. 
In particular, the optimal cost functional can be represented via two coupled stochastic Riccati 
equations. Some optimality conditions for MLQ problem is therefore obtained using the equivalence 
among MLQ, RLQ and OT problems. 

Keywords. Mixed linear-quadratic optimal control, optimal stopping, maximum principle, Riccati 
equation. 



1 Preliminary and Problem Formulation 

Let T > be given and (fi, J^, P, F) be a complete filtered probability space on which a one dimensional 
standard Brownian motion W{-) is defined with F = {J't}t>o being its natural filtration augmented by 
all the P-nuU sets. We consider the following stochastic controlled system: 

( dXi{t) = [Ai{t)Xi{t) + Bi{t)u{t)]dt + [C\(t)Xi(t) + Di{t)u(t)]dW{t), te [0,t), 
(1-1) < dX{t)^[A{t)X{t)+B{t)u{t)]dt+[C{t)X{t) + D{t)u{t)]dW{t), te[T,T], 

[ Xi{0) = XI, X{t) = K{t)Xi{t - 0), 

where Ai{-), Bi{-),Ci{-), Di(-), A{-), B{-),C{-), D{-) and A'(-) are given matrix-valued functions of com- 
patible sizes. In the above, X{-) = {Xi(-), X2{-)) is the state process, taking values in R", which is 
decomposed into two parts, Xi{-) is valued in M"' {i = 1,2, ni+n2 = n), u{-) is a (usual) control process 
taking values in some set U C R™, and t is an F-stopping time. From the above, we see that the part 
Xi{-) of the state process X{-) starts to run from xi E R"^ a.t t — 0. The total system will start to 
run at a later time t — t, with the initial state X(t) depending on Xi[t — 0). Besides the usual control 
m(-), the stopping time r will also be taken as a control. The above state equation can be interpreted as 
follows: we let ^i(-) represent the dynamics of some basic project whereas X2{-), initialized at the time 



*This work is supported in part by RGC Grants GRF521610 and GRF501010, and NSF Grant DMS-1007514. 



T, represents an additional or an auxiliary project. It is notable that the initial value of X2{-) depends 
on Xi{t), the value of the first component of the state at time r. Some real examples are as follows. 

Example 1.1. (Urban Planning) Let Xi{t) denote some quantity of the dynamic value of some 
basic infrastructure investment in urban planning (for example, the transportation network, systematic 
pollution protection, and so on) at time t, while X2{t) denotes the quantity of the real-estate property in 
urban planning, at time t>T, where t G (0,r) is the time moment at which some basic infrastructure 
has been set. Note that the construction of basic infrastructure will still be continued (although it will 
be less intensive) after r. It follows naturally the real estate property should depend closely on Xi{-). 

Example 1.2. (Applied Technology) Let Xi{-) represent the capital investment of some high-tech 

company in the phrase of primary research and development (R&D) while let X2{-) denote the capital 
investment in the phrase of technology marketing and product promotion, etc. Of course, X2{-) will 
depend on the competitive ability of the product which in turn depends on the technology ability in basic 
research Xi{-). 

Note that if A{-), B{-), C(-), -D(-) are of the following form: 

then on the time period [r, T], the part Xi{-) will be completely stopped and only the part X2{-) will be 
running. 

Now we introduce the following quadratic cost functional: 

J{xi ;u{-),t) = Ie{ f [ ( Qi {t)Xi (t), {t)) + { Ri (t)u(t) ,u{t))]dt + (t)Xi (r) , X, (r) ) 
(1-2) ^ '-^^ ^ 

+ ^ [ ( Q{t)X{t),X{t) ) + ( R{t)u{t), u{t) )]dt+{ GX{T),X{T) ) }, 

where Qi{-),Ri{-),Q{-),R(-), and G'i(-) are symmetric matrix-valued functions, and G is a symmetric 
matrix, of suitable sizes. Roughly speaking, our optimal control problem is to minimize J{xi;u{-),t) 
over the set of all admissible controls (!/(•), t). We now make our problem formulation more precise. 

For Euclidean space M", we denote by ( • , • ) its inner product and | • | the induced norm. Next, let 
j^mxn ^jjg gg|. all m X n real matrices, S" be the set of all n x n symmetric real matrices, and for 
any M = {rriij) e M™^", stands for its transpose and let 

j_ 

|M| = (^TO^j.)', VMeM"*^". 

For X = R",R"^™, etc., let 

L'^{a,b;X) 
L°°{a,h;X) 



= [f ■.[a,b]^X\ 



/(•) is measurable, 
I /(•) is measurable, 

/(•) is continuous \ 



[ |/(t)|2rft<+oo}, 

/a 

esssup \f{t)\ < oo >, 
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and 



= U : r2 -> X I ^ is J"t-measurable, E|^|'' < cx) L p > 1, 



= |e e I esssup|e(a;)| < ooj, 




L^ia,b;X) = {f{-)eLl{a,b;X)\ esssup w)| < oo}, 



(t,cj)e[a,ft]xi2 




te[a,6] 



L^(l];C([a,6];X)) = f/(.)ei^(a,6;X)|E[ sup |/(i)H < +^}, p > 1. 



'- te[a,6] 



In the definition of Cr{[a, b]; X)), 1 1-^ f{t) is continuous means that 

limE|/(0-/(to)r-0, Vioe[a,&]. 



Now, let us introduce the following sets: 



U[0,T] = |u : [0,r] X 17 ^ t/ I u(-) e i|(0,T;M™)|, 
r[0,T] = (r : ^ [0,r] I T is an F-stopping time}. 



Hereafter, U C M™ is assumed to be convex and closed. Any u{-) G U[0,T] is called a regular admissible 
control while r € T[0,r] is called an admissible stopping time. Under some mild conditions, for any 
xi € R"S and {u{-),t) e U[0,T] x T[0,r], dTT]) admits a unique strong solution X{-) = X{- ■,xi,u{-),t), 
and the cost functional (|1.2I) is well-defined. Having this, we can pose the following problem. 



Any {u{-), f) G U[0, T] xT[0, T] satisfying the above is called an optimal control pair, X{-)=X{- ; xi, u(-), f) 
is called the corresponding optimal trajectory, {X {■) , u{-) , f) is called an optimal triple. In the above, 
MLQ problem stands for mixed linear- quadratic problem, in which, one has a usual control u(-) mixed 
with a control t of stopping time. We have the following points to the above MLQ problem formulation. 

• A special feature of Problem (MLQ) is that in minimizing the cost functional, one needs to select 
a regular control u(-) from U[0, T] and at the same time, one has to find the best time r to initiate 
or trigger the whole system. Note that the initial value X{t) of X{-) at r depends on the value of 
Xi{t — 0), which in turn depends on the regular control on [0,t). From this viewpoint, the Problem 
(MLQ) is some kind of combination of a usual stochastic optimal control and an optimal stopping 
time problems. Similar problems have been investigated in literature, including 0ksendal and Sulem 
[5] where some optimal resource extraction optimization problem was addressed. By applying the 
dynamic programming method, the optimal policy can be characterized by some Hamilton- Jacobi- 
Bellman variational inequalities, see Krylov [5], for relevant treatment. In contrast, here, we aim to 
investigate the problem by the variational method which could lead to Pontryagin type maximum 
principle. 



Problem (MLQ) For given xi G ]R"i , find a (u(-), r) G U[0, T] x r[0, T] such that 



J{xi;u{-),t) 



inf 

{u(-),T)eU[O.T]xT[0,T] 



J{xi;u{-),t) = V{xi). 
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Consider the following simple but illustrating example, from which we can see the significant dif- 
ference between Problem (MLQ) and other relevant ones when applying the possible perturbation 
method. Suppose the controlled state equation is given by 

dXi{t) = aiXi{t)dw{t), t e [o,r], 

dX2{t) - a2X2{t)dW{t), t e [t,T], 
X2(t) = KX.ir), 

where ai,a2,K are some constants. Let X{-) = {Xi{-), X2{-)) be the solution corresponding to f. 
Introduce a perturbation on f of the form: r'' = f + pr, p > 0, with t being another stopping time. 
Let the solution corresponding to be XP{-) = (Xf (•), (•))■ Then (■) = Xi{-) is independent 
of p, and for i e [t'',T], 

X^it) - X2{t) = k[x^{tP) ~~ Xi(f)) + ^ "2 {XP{s) - X2{s)) dW{s) - J_ a2X2is)dWis). 

The presence of the term J- a2X2{s)dW {s) makes the first-order Taylor expansion in convex 
variation failed to work. This is mainly due to the fact that 

\ 2 p 
a2X2{s)dW{s)\ =E / {a2X2{s)fds, 




which, in rough sense, suggests /J a2X2{s)dW{s) be of order y^p instead p. On the other hand, it 
is not feasible to apply the second-order Taylor expansion to introduce the second-order variational 
equation (as suggested by Peng [9], Yong and Zhou [13] etc.). This is mainly because the time 
horizon on which X2{ ) is defined depends on the selection of r, thus the spike variation method 
cannot be applied here either. 

• Problem (MLQ) also differs from the well-studied stochastic impulse control, since as the time 
passes r, instead of having a jump for the state as a usual impulse control does, our controlled 
system changes the dimension of the state (from Xi{-) to -'^"(•))- 

In summary, the involvement of t into the control variable makes the Problem (MLQ) essentially 
different from other classical optimal control problems, and the standard perturbation jointly on (m(-), f ) 
is not workable directly. Keep this in mind, in this paper, we take the following strategy to study Prob- 
lem (MLQ): we first connect the Problem (MLQ) into some random-duration linear quadratic (RLQ, for 
short) optimal control problem, and an optimal time (OT, for short) problem to the associated Riccati 
equations. By Problem (RLQ), we can obtain some necessary condition for the regular optimal control 
it(-); by Problem (OT), we can obtain some necessary condition satisfied by the optimal time f. Next, 
by solving Problem (RLQ) and Problem (OT) consecutively, we can solve the original Problem (MLQ). 



The rest of this paper is organized as follows. In Section 2, we get a stochastic maximum principle 
for a little more general two-stage random-duration optimal control problems. Based on it. Section 3 is 
devoted to a study of the random-duration linear quadratic optimal control problems. The state feedback 
optimal control is derived via some stochastic Riccati-type equations and the optimal cost functional is 
also calculated explcitly. In Section 4, an equivalence between Problem (MLQ) and Problems (RLQ)- 
(OT) is established. In Section 5, for the case of one-dimension with constant coefficients, we characterize 
the optimal time f. 
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2 Random-Duration Optimal Control Problem 

In this section, we consider the fohowing controlled stochastic differential equation (SDE, for short) 



(2.1) 



' dXi{t) = b^{t,Xi{t),u{t))dt + a^{t,Xi{t),u{t))dW{t), t€[0,T), 

dX{t) = b{t. X{t), u{t))dt + a{t, X{t),u{t))dW{t), t € [r, T], 
. Xi(0)-a;i, X(t) = $(t,Xi(t)), 



where r € T(0, T] is some fixed stopping time and u{-) € W[0,T] is an admissible control. The cost 
functional is 

(2.2) J-(a;i;^0)=E[^"5'(i,^i(i),^O)rfi + j[^^5(i,^(t),Mi))rfi + /i'(T,Xi(T))+M^(r)) 

Consider the following random-duration optimal control (ROC, for short) problem: 



Problem (ROC) For xi e M"i and r € r[0,r], find a u(-) e U[Q,T] such that 



(2.3) J-(xi;w(-))= M J^{xrM-))^V^{x^). 

The following basic assumptions will be in force: 
(H2.1) Let 

6,0- : [0,r] X R" X i7 X n -s- M", &\ct^ : [0,r] x R"i x C/ x O ^-K"!, 
$ : [0, T] X X f2 M" 



be measurable such that 



(t, Cj) I— > (6(t, X, U, Cj), (T(t, X, U, Cj)), 

(i,w) I— >■ (6"'^(i, X, u, w), (7"'^(i, X, u, w), $(i, xi, w)), 
are progressively measurable, 

(x, u) ^ {b{t, X, u, cj), a{t, X, u, w)), 

(xi,u) (6^(f,xi,u,a;),cr^(f,xi,M,a;),$(f,xi,a;)), 

are continuously differentiable, and for some constant L > 0, 

\b4t,x,u)\ + \bu{t,x,u)\ + \bl^{t,xi,u)\ + \al^{t,xi,u)\ + \<^^,{t,Xi)\ < L, 

V(t, X, u) e [0, T] X R" X U, a.s. , 

and 

\b{t,0,u)\ + \a{t,0,u)\ + \b\t,0,u)\ + \a\t,0,u)\ + |$(t,0)| < L, {t,u) G [0,T] x U, a.s. 



(H2.2) Let 

5 : [0, T] X M" X f/ X O -s- R, 5^ : [0, T] x R"i x ?7 x R, 
h-.WxQ^R, /i^ : [0,T] X R"i X -s- R 
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be measurable such that 

{t,u) i~-> g{t,x,u,uj)), {t,u) i-> g^{t,x,u,u) 
are progressively measurable, 

{x, u) I— >■ g{t, X, u,uj), {xi, u) H- 5- g^{t, xi, u,uj), 

{xi,u) {b'^{t,xi,u,uj),a^{t,xi,u,uj),^{t,xi,uj)), 
are continuously difFerentiable, and for some constant L > 0, 

\g4t,x,u)\ + \guit,x,u)\ + \h4x)\ < L{1 + \x\ + \u\), 
\gl^it,xi,u)\ + \gl{t,xi,u)\ + \hl^{t,xi)\ < L{1 + \xi\ + \u\), 

W{t,x,u) e [0,t] X R" X U, 

and 

\git,0,u)\ + \g\t,0,u)\ + \h'{t,0)\ < L, V(i,u) G [0,T] x U. 

By some standard arguments, we see that under assumptions (H2.1)-(H2.2), for any xi S R"'^ and 
(■u(-),r) S Li[0,T] X T[0,T], the state equation (|2.ip admits a unique solution X{- ■,xi,u{-),t) and the 
cost functional J'^ {xi;u{-)) is well-defined. Therefore, Problem (ROC) makes sense. Suppose {X{-),u{-)) 
is an optimal pair of Problem (ROC), depending on (xi,t). For any v{-) E U[0,T], denote 

"(•) = vi-) - u{-). 

By the convexity of [/, we know that 

uP{.) = u{.) + pu{-) = (1 - p)u{-) + pv{-) e U[0, T], Vp e (0, 1). 

By the optimality of {X{-),u{-)), we have 

^^ J^xuu''{■))-.^ix^■,ui■)) 

P ~ 

Making use of some similar arguments in [7], we have the following result. 

Lemma 2.1. Suppose (H2.1)-(H2.2) hold. Let {X{-),u{-)) be an optimal pair of Problem (ROC). 
Then 



(2.4) 



/ [gl (i, Xi (t) , S(i))6 (t) + gl (i, X, (t), u{t))uit)] dt + hl (r, X^ (r))Ci (r) 

[<?,(t, X{t), uitmt) + gu{t, X{t),u{t))n{t)\ dt + h.,{X{TmT)\ > 0, 
where ^(•) = (Ci(-)iC2(')) ^® solution to the following variational system: 

' = [fei,(i,Xi(t),ii(i))ei(i) + fei(t,^iW,^(t))7/(t)]di 

^al^{t,X,{t),u{t))^At) + al{tai(t),u{t))u{t)]dW{t), [0,t), 

dC(t) = [6,(i,X(t),M(t))e(t) +&„(<, X(t),M(t))M(i)]di 

^a,{t,X{t),u{t))^{t) + au{t,X{t),u{t))u{t)]dW{t), te [t,T], 
I 6(0) =0, e(r) = $.,(T,Xi(T))a(r). 



The following is a Pontryagin type maximum principle. 
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Theorem 2.2. Suppose (H2.1)-(H2.2) hold. Let {X{-),u{-)) be an optimal pair of Problem 
(ROC). Then the following two backward stochastic differential equations (BSDEs, for short) admit 
unique adapted solutions {p{-),q{-)) and {pi{-),qi{-)): 



(2.5) 



dp{t) = - [b,{t, X{t),u{t)fp{t) + ajt, X(t),u{t)fq(t) 

-gS,X{t),u{t)f]dt + q{t)dW{t), te[T,T], 
p{T) = ~h,{X(T)f, 



(2.6) 



' dpi{t) = -[bl^{t,Xi{t),uit)fp,{t) + al^{t,X,{t),uit)fq,it) 
-gl^{t,Xi{t),u{t)f]dt + qi{t)dW{t), tG[0,T 
- Pi{r) = -hi^{T,X^{T)f + ^,,{T,Xi{r)fp{r). 
Moreover, the following variational inequalities hold 



(2.7) 



p,{tfbi{t,X,{t),u{t)) + q,{tfalit,X,{t),u{t)) - gl{t, X,{t),u{t))\ [v - u{t)] < 0, 

t G [0,r), V €U, a.s. , 
p{tfbu{t, X{t), u{t)) + q{tfau{t, X{t),uit)) - 5„(i, X{t),u{t))\ [v - u{t)] < 0, 

t e [t,T], V €U, a.s. 



Proof. Applying Ito formula to {pi{-),^i{-)) and {p{-),^{-)), respectively, we have 



E 



( -hi (r, Xi {T)f + (r, Xi {T)fpiT), a (r) ) = E ( p, (r) , 6 (r) ) 



Jo L 



(i, Xi , ufpi + al^ {t, Xi , ufqi - gl^ {t, Xi , uf] , ^ ) 



+ {Pi,b^^{t,Xi,u)^i +b^{t,Xi,u)u) + {qi,a^^{t,Xi,u)^i + (T^{t,Xi,u)u) 



E / \gl^{t,X,,u)^, + {bi{t,X,,ufp,+ai{t,X,,ufqu 
Jo L 



dt 



u) 



dt. 



and 



E ( -h,{X{T)f, ^{T) ) = E{p{T),aT) ) 

= e{ {p{t), C(r) ) + ^ [{-[bS, X, ufp + aS, X, ufq - g.{t, X, uf] , ^ ) 

+ {p, bx{t, X, u)^ + bu{t, X, u)u ) + {q, a^it, X, u)^ + (7„(t, X, u)u ) j dij 

= e{ (p(r), (r, Xi(r))Ci M ) + X, u)£, + ( 6„(t, X, w)^p + CT„(i, X, M)^g, w ) ] di}. 

Then, we obtain 



< E 



^ (t , , ^2)a + gl (i , Xi , ^2)u] + fti^ (r, Xi (t))Ci (t) 



+ ^ [g^it, X, u)^ + gu{t, X, u)u\ dt + h^{X {T))^{T)} 
= e{ ^ ( -bUt, Xi,ufpi - alit, Xi,ufqi + gUt, Xi,uf, u)dt + {^,, (r, Xi(r))^p(T), ^(t) ) 
+ ^ ( -bu{t, X, ufp - au{t, X, ufq + gu{t, X, uf,u)dt- {p{t), (r, Xi(r))a (r) ) } 
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= E| {~bl{t,Xi,uypi-at{t,Xi,uyqi+9tit,Xi,uy,v-u)dt 

+ J {-bu{t,X,u)^p-au{t,X,u)'^q + gu{t,X,u)'^,v-u)dty 
Note that v{-) £U[0,T] is arbitrary, we therefore have (|2.7p . 

3 Random-Duration Linear Quadratic Problem 

X T[0,T], we consider the foUowing controlled linear system: 



For any {xi,t) e R"i 
(3.1) 



dXi{t) = [Ai{t)Xi{t) + Bi{t)u{t)]dt + [Ci{t)Xi{t) + Di{t)u{t)]dW{t), te[0,- 
dX{t) ^ [A{t)X{t) + B{t)u{t)\dt + [C{t)X{t) + D{t)u{t)\dW{t), t€ [t,T], 
XiiO) = xi, X(r) = X(t)Xi(t-0), 



with quadratic cost functional as follows 
1 



(3.2) 



2 1./0 



[ ( Qi it)Xi it) ,Xi{t)) + { Ri {t)u{t), u{t) )]dt+{Gi (r)Xi (r) , Xi (r) ; 



; Q{t)X{t), X{t) ) + { R{t)u{t),u{t) )]dt+{ GX{T), X{T) ) }. 



Now we pose the following problem. 



Problem (RLQ) For given (a;i,r) e M"i x r[0,T], find a u(-) e U[Q,T] such that 



(3.3) 



r{xi-u{-))= inf r{xi;u{-)) = V^{xi). 
u{-)eu[o.T] 



We call the above a random- duration linear quadratic (RLQ, for short) problem. For the above 
problem, we introduce the following hypothesis. 



(H3.1) The following holds: 



X n 
7)71 X m 



Ai(.),Ci(.) e L^(o,r;R"^x"^), A(.),C(.) e L^{Q,T;] 
B,{-),D,{-) e i^(0,r;R"iX"), Bi-),D{-) e i^(0,r;R"><™), 

gi(-)eir(0,r;§"^), g(-) ei?°(0,T;S"), i?i(-),i?(-)ei?°(0,T;S™), 

[ Gi(-)eCK[0,T];L|?^(f];§"^)), GeL^(l};§"), if (•) G CK[0, T]; L|?^(l]; 
Moreover, for some (5 > 0, 



(3.4) 



Riit)>SIn„ Rit)>6In, 
Qi(t),Gi(t) > 0, Q{t)>0, G>0, 



t e [0,T], a.s. 



It is clear that under (H3.1), for any given {xi,t) G M"^ x T[0,T], the map u(-) H' J(xi;m(-),t) is 
convex and coercive. Therefore, Problem (RLQ) admits a unique optimal control u{-) € U[0,T]. Now, let 
(X(-),u(-)) be the optimal pair of Problem (RLQ), depending on (xi,t) e ]R"i x T[0,r]. By Theorem 
2.2, on [t,T] the optimal pair {X {■),€,{■)) satisfies the following: 



(3.5) 



dX(t) = [A{t)X{t) + Bit)uit)]dt + [Cit)X{t) + D{t)u{t)]dW{t), 
dp{t) = -[A{tfp{t)+C{tfq{t) - Q{t)X{t)]dt + q{t)dWit), 
p{T) = ^GX{T), X{t) = K(t)X^{t), 
, B{tfp{t) + D{tfq{t) - R{t)u{t) = 0. 
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The above is a coupled FBSDE in random duration. To solve it, we let 

p{t) = -P{t)Xit), t€[0,T], 

for some P(-) satisfying 

f dP{t) = T{t)dt + A{t)dW{t), t G [0, T], 
\ P{T) = G. 

Then apply Ito's formula, we have 

( - A^PX + C'^q - QX)dt ~ qdW = -dp = d{PX) 

= ^^rX + P{AX + Bv) + A{CX + Dv)^dt + ^AX + P{CX + Du)^dW 

= {(r + PA + AC)X + {PB + AD)u^dt + {(A + PC)X + PDu^dW. 



Hence, 
Then 

This implies 



q = -{A + PC)X - PDu. 

Ru = B'^p + D^q = -B'^PX - D^(A + PC)X - D'^ PDu. 



u = -{R + D^PD)-'^{B'^P + D^A + D^PC)X. 
Substituting u into the expression of the above q yields 

q = -(A + PC)X + PD{R + D^PD)-^{B^P + D^A + D^PC)X. 



Moreover, 



= {A^P + Q)X - C'^q +{T + PA + AC)X + [PB + AD)u 

= {T + PA + A^P + AC + Q)X 

-C'^ [ - (A + PC)X + PD{R + D^PD)-\B^P + D^A + D^PC)X 
-{PB + AD){R + D^PDy^ [B^P + D^A + D^PC)X 

= {t + PA + A^P + C'^PC + AC + C'^A + Q 

-{PB + AD + C'^PD){R + D^PD)-'^{B'^P + A + D'^PC)^X. 



Therefore, we take 
r = 



(3.6) 



PA + A^P + C'^PC + AC + C'^A + Q 
-{PB + AD + C^PD){R + D^PD)-^ {B^P + D^A + D^PC) 

Consequently, the corresponding Riccati equation reads 



(3.7) 



dP = - PA + A^P + C'^PC + AC + C^A + Q 

-{PB + AD + C'^PD){R + D'^PD)-^ {B'^P + D^A + D'^ PC) 
+AdW, te[0,T], 
P{T) = G, R + D'^PD>0. 



dt 
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(3.8) 



On [0,t] the optimal pair {Xi{-),ui{-)) satisfies 

' dXi=[AiXi+BiR-^Blpi+BiR~^Djqi]dt+[CiXi + DiR-^B^Pi + DiR-^Djqi]dW{^^ 
dpi = - [Ajpi + CUi - QiXi] dt + qidW{t), 
Xi(0) pi(r) = -[xirf P{t)K{t) + Gi{t)\Xi{t), 

{Riui = B^pi+Dfqi. 

Similar to the above, the corresponding Riccati equation is 



(3.9) 



dPi = - 



PiAi + AjPi + CjPiCi + AiCi + Ci^Ai + Qi 

-{PiBi+AiDi + C^PiDi){Ri+DjPiDi)-^{BlPi+D'[Ai+D'[PiCi) 



dt 



+AidW, te[0,T], 

[ P^{T)^K{TfP{T)K{T)+G^{T), 



Ri+D{PiDi > 0. 



Following [T^, we know that under (H3.1), Riccati equations p.7p and p.9p admit unique adapted 
solutions on [0, T] and [0, r] respectively. The following is a verification theorem. 

Theorem 3.1. Let (H3.1) hold. Let P{-) and Pi{-) be the solutions of Riccati equations p.7p and 
p.9|) respectively. Then Problem (RLQ) has an optimal control with state feedback form as follows: 



(3.10) 

where 

(3.11) 



u{t) 



-'^{t)X{t), te[T,T], 
-*i(t)Xi(t), te[0,T), 



*(<) = [R{t) + D{tfP{t)D{t^ \^B{tfP{t) + D{tfP{t)C{t) + D{tfA{t) 
*i(i) = \Ri{t) + D,{tfPi{t)Di{t)\ \Bi{tfPi{t) + Di{tfPi{t)Ci{t) + Di{tfAi{t) 



Moreover, the optimal value of the cost functional is given by 

1 



(3.12) 



J^xi;u{-)) = - (Pi(0)xi,xi) = V^xi). 



Proof. For any u(-) g U[0,T], let X{-) be the corresponding state process. Applying Ito formula to 
{P{-)X{-),X{-)) on the interval [t,T], we obtain (let F be defined by (|3Jl) ) 

E[{GX{T),XiT))-{PiT)X{r),Xir))] 



E 



^ (^{{T + PA + A'^P + C'^PC + AC + C^A)X,X} 

+2 {{B'^P + D^A + D'^PC)X, u) + { D'^PDu, u) ]dt 



I y" [2 ((B^P + D^PC + D^A)X, u)-{ QX, X) + { D^PDu, u ) 

+ {{PB + C'^PD + AD){R + D'^PD)-\b'^P + PC + D^A)X, AT ) 
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It follows that: 



r{x,;ui-)) - -E{P{T)XiT),XiT)) 



-E 



i-T 



-E 



( QiXi , ) + ( i?i 7. > ) rfi + ( Gi (r)Xi (r) , (r) ) 

(^{QX,X) + {Ru,u)'^dt+ {GX{T),X{T)) 

{GX{T),X{T))+ J [2{{B'^P+D'^PC+D^A)X,u}~{QX,X} + {D^PDu,u} 
{{PB + C^PD + AD){R + D'^PD)-^{B^P + PC + D'^A)X, X)]dt^ 

( ( QiXi , ) + ( i?i u, u ) ) + ( Gi {t)Xi (t) , Xi (t) ) 

J ((i? + D'^PDy'^iB^P + D'^PC + D'^A)X, {B^P + D'^PC + D^A)X ) 
-2 ((S^P + D^PG + DA)X, M ) + ((P + D^PD)u, u ) ) dij 

( ( QiXi , Xi ) + ( Piu, ) ) + ( Gi (t)Xi (r) , (r) ) 

{R + D'^PDy u+ {R + D'^PDy\B'^P + D'^PC + D'^A)X dij. 

Therefore, we have 

J"(a;i;u(-)) = ( ( Qi^i, Xi ) + ( Pi^/, u ) )dt + ([Gi(t) + i^(T)^P(r)i^(r)]Xi(r), Xi (r) ) 

{R + D'^PD)i u + {R + D^PD)''^{B'^P + D'^PC + D^A)X ^dtj. 
Next, applying Ito formula to {Pi{-)Xi{-),Xi{-)) on the interval [0,t], similar to the above, we have 

J"(xi;7/(-))-^Ie{(Pi(0)xi,xi) 

+ / {Ri+D'[PiDi)i\u+{Ri+D{PiDi)-\BlPi+DfPiCi+DfAi)Xi] dt 
Jo L J 

-1 2 

(R + D^PD)^ u+ {R + D'^PUy^B'^P + D'^PC + D^A)X dt\ 
= ^Ej (Pi(0)xi,a;i) + ^ |(Pi + ZJf PiP>i)-i(u + ^iX^^dt + ^ |(P + P>^PP>)-i(u + ^'X)|2di}. 
Then our conclusion follows. □ 



4 The Equivalence of Control Problems 



Under (H3.1), Problem (RLQ) admits a unique optimal pair which can be represented by p.lOp . In this 
section, we will establish some connection between Problem (MLQ) and Problem (RLQ). To this end, we 
denote the solution to Riccati equation p.9p on [0,t] by (P]^(-), A[(-)), emphasizing its dependence on 
T via the terminal condition. It is clear that r i— >■ P[{s) is continuous. Therefore, the following problem 
makes sense: 
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Problem (OT) For given xx £ M"^ , find are T[0, T] such that 



(4.1) 



{P^{0)xx,xi) = inf {P^{0)xi,xi) 

rer[0,T] 



Any f G T[0,T] satisfying the above is called an optimal time of Problem (OT). Note that in the case 
rii = 1, if f is an optimal time for some S M \ {0}, then it is an optimal time for all G M \ {0}. On 
the other hand, since there might not be some kind of monotonicity of the map r i— )■ Pi{s), optimal time 
f may not be unique. Now we establish the equivalence between Problem (MLQ) and Problems (RLQ) 
and (OT). 

Theorem 4.1. Suppose (H3.1) holds and (■u(-),f) is an optimal control pair of Problem (MLQ). 
Then the optimal control u(-) can be represented by 



(4.2) 
where 



m = 



-^{t)x{t), ie[f,T], 

-W^{t)Xx{t), tG[0,f), 



(4.3) 



^r(i) = R{t) + D{tfP{t)D{t) B{tfP{t) + D{tfP{t)C{t) + D{tfA{t) 



mt) - i?i(t) + D,{ty Pi{t)Dx{t) B,{ty Plit) + D{tY Pi{t)c,{t) + D^ity K[{t) 



Moreover, the optimal value of the cost functional for Problem (MLQ) is given by 



(4.4) 



J(xi;i2(-),f) = i(Pf(0)xi,a:i), 



with T being an optimal time for Problem (OT) to Riccati equation (|3.7I) and (13. 9p . 
Proof. Suppose {u{-),f) is an optimal pair of Problem (MLQ), that is 

J(xi;M(-),f) = inf J{xi;u{-),t). 

iu{-),r)£UlO,T]xriO,T] 

We fix f. Then the above implies 

rixi;u{-))= inf r(xi;u{-)). 

u{-)£UlO,T] 

Then it follows from Theorem 3.1 that u{-) admits representation ()4.2|) with ^'(•) and ^'[(•) given by 
and 

r(xx;u{-)) = ^{P!{0)x,,x,). 
Next, for any stopping time r G T[0,T], we can construct a control u G U[0,T] satisfying 

u{t) 



-^it)X{t), <G[r,r], 
-^l{t)Xi{t), tG[0,T), 



where X(-), Xi{-) are defined in similar way to X{-), Xi{-) but replacing r by r. Following the similar 
arguments, we can prove 



rixuui-))^ inf J-(xi;u(.)) = -(P:(0)xi,xi) 
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Then 



- (Pf(0)a;i,xi) = r{xv,u{-)) = J{xi;u{-),f) 

= inf J(xi;u(-),t) < J(xi;u(-),t) = I- (Pt(0)xi,Xi) . 

(«(-),r)ew[o,T]xr[o,T] V ^' w> y - V V y 2^1^^"^' 

That is, f solves the Problem (OT). Hence the results. □ 

Next, we consider the case of deterministic coefficients. More precisely, we introduce the following 

assumption. 

(H3.1') The following holds: 

Ai(-).Ci(-) e L°°(0,r;R"i^"i), A(-),C(-) € L°°(0, T; R"^"), 
Bi{-),Di{-) e L°°(0,r;R"i^"), B{-),D{-) e L°^(0,r;R"^'"), 

(4-5) <( Oi(-) e i°°(o,r;§"i), g(-) e L°°(o,r;§"), 

e L°°(0,T;S'"), e L°°(0,T;S'"), 

I Gi(-) e C([0,T];S"i), GeS", i^O) e C([0,T];M"^"^). 



Moreover, for some ^ > 0, 
(4.6) I 



Rl{t)>dln,, R(t)>5In, 

Qi{t),Gi{t)>0, Q{t)>0, G>0, 



te [0,T]. 



We have the following result. 

Proposition 4.2. Let (H3.1') hold. Let P(-) solves 



(4.7) 
and Pi'(-) solves 
(4.8) 



P + PtI + tI'^P + C'^PC + Q- [PB + C'^PD){R + D'^PD)'^ [B'^P + £)'^PC) = 0, 

ie [o,T], 

P(T) = G, i? + D'^PD > 0, 



P[ + P[Ai + AlPl + C^P[Ci + Qi 

-(PfSi + C];P[D,){R, + DlP[Di)-^ {BjPl + DjP[Ci) =0, t€ [0, r], 
[ Pf (r) = i^(r)^P(r)/ir(r) + Gi(r), Pi + DjP^Di > 0, 



with r e (0,r). Then Problem (MLQ) has an optimal control pair {u{-),f) G U[0,T] x [0,r] where 
(4.9) 



witii r being deterministic and 



(4.10) 

Moreover, 
(4.11) 



^{t) = R{t) + D{tfP{t)D{t) B{tfP{t) + D{tfP{t)C{t) 



^lit) = R,{t) + D,{tf Pl{t)D^{t) B,itfP[it) + DitfP[{t)Ciit) 



{P^{0)xuxi)= inf {P({0)xuxi). 

re[0,T] 
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Proof. In the current case, Riccati equation (I3.7P becomes (|4.7p . On the other hand, if we still denote 
(Pf (•), A[(-)) to be the adapted solution to the Riccati equation (I3.9p . then when r = r e (0,r) is 
deterministic, one has AJ(-) = and P[{-) satisfies deterministic Riccati equation (|4.8p . It is clear that 
if f e [0, T] satisfies (|4lT|) . then 



2 rer[o,T] 2 

This proves our result. 

For the convenience below, we state the following problem. 



(4.12) 



Problem (DOT) Find f e [0,T] such that 

(P[(0)a;i,a;i) = 



inf (P[(0)a;i,a;i; 

re[0,T] 



By Proposition 4.2, we see that the optimal time f in Problem (DOT) solves Problem (MLQ). More- 
over, in general, the optimal time for Problem (DOT) depends on the Riccati equation P[{-) and the 
initial condition xi. The following example makes this clear. 

Example 4.3. Let ni — 2, n2 — I, and let 



A-- 




Q 



Qi 





1 , i? = l, G 









, i?i — 1, Gi 










c= 


( 




































(: 













9 


91 





)• 












Di = 



K-- 




with a G M, g,gi G (0,cx)). Then for any r G (0,T), we have the state equation 

' Xl{t)=Xf{t), iG [0,r), 

XUt)^uit), tG[0,r), 
X2{t) =aX2{t)+u{t), tG[r,r], 
Xi(0)=.Ti, X2{r)=Xl{r). 



The cost functional reads 

J{xi;u{-),r) 
In this case, we have 



\u{t)\'dt + g,\Xl{r)\'+ r \u{t)\'dt + g\X2{T)\'}. 




P{t) =(00 

P2(<) 
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with P2(-) solves the following Riccati equation: 



P2{T)=g. 



This equation admits a unique solution P2{-)- We claim that 



(4.13) 
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Note that if a > 0, then 
and if a < 0, then 
Hence, 



l + 5(T-t)' 

g(^g2a{T-t) _ 1) + 2a > 2a > 
^(g2a(T-t) _ 1) + 2a < 2a < 



te[r,T], a = 0. 



Next, the Riccati equation for Pi{-) reads 

f A (t) + Pi {t)Ai + A{Pi {t) - Pi {t)MPi [t) = 0, t e [0, r] 
1 Pi(r) = Gi(r), 



where 



M BiBl = 




1 



9i+P2{r) 




Gi{r) = K'^P{r)K + Gi = 
We now solve the above Riccati equation by a method found in |6]. To this end, let 

Pi{t) = Pi{t) - Gi{r), ie[0,r]. 

Then (suppressing t and r) 



Pi=Pi 



PiAi+AiPi -PiMPi 
{Pi + Gi)Ai + Aj{Pi + Gi) - (A + Gi)M{Pi + Gi) 
Pi{Ai - MGi) + {Ai - MGifPi - PiMPi + Gi^i + A^Gi - GiMGi 



Note that 



MGi 



\ / gi+P2(r) 



1 











0, 



9i+P2{r) 



1 



9i + P2{r) 




y V 

Hence, Pi{ ) should be the solution to the following Riccati equation: 

I Pi{t) + Pi{t)Ai+AlPi{t)^Pi{t)MPi{t) + -giJ = {), ie[0,r]. 
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where 



Next, we let 



J = 



1 

1 



91 = 91 +P2{r). 



A 



-9iJ -Al 



1 





1 








\ 













-1 

















\ 







-1 





/ 



Then according to [6], we have the fohowing representation of the solution Pi(-): 



AW = - (0,/)e 



A{r-t) 



(0,/)e 



A{r-t) 



te[0,r] 



as long as the involved inverse exists. We now calculate e-^^'' Direct computation show that 

, a:^ = A^ = 0. 



A' = 



/ -1 \ 

ffi 1 

31 

\ / 



f 91 1 \ 



-gl -51 

\ y 



Hence, 



(r - tf 



A' 



( i + Mi^ 

9l(r-tf 

V -9i{r-t) 



{r-t) 
1 



6 



-{r-t) 



-gi{r-t) i-Mr^ Mr^ 



Then 



(0,/)e 



A(r-t) 



(0,/) 







$11 $12 
$21 ^22 



6 



$11 $12 
$21 ^22 



1 / 



-(r-<) 1 



Since 



we have 



det 



-(r-i) 1 



l-f(r-i)^ f(r-t)2 



-{r-t) 



On the other hand. 



(0,/)e-^('-*) 



Hence, 

PiW--f(0,/)e-^^'^-*^ 



(0,/) 



$11 $12 
$21 *22 



= $21 = 



-9i{r-t) 



6 5i(r-0 



(0,/)e 



1 

3 + gi(r-i)3 V ('■-O l-f(r-i)= 
3 + gi(r-t)3 I -gi(r-i) -5i(r-0' 



6 ffll'^-t) 



= -*22^*21 

-9i{r- t) 

9i{r-t) 
3 + 5i(r--i)3 



-5i(r-i)2 3 
3 3(r-t) 
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Consequently, 



3[gi+P2{r)] 



1 r-t 



3+[gi+P2{r)]{r-tr \ r-t {r-tf 
which is positive definite on [0, r). Hence, 

3[ffi + P2(r)] 



t G [0,r], 



V{r,xi) = {P[{0)xi,xi) = 
'i[gi+P2{r)] 



3+bi+-P2(r)]r3 
■[x\+rxl]\ 



{x\r+2rxlxl+r'{xlf 



■i + [gi + P2ir)y 

Clearly, for different xi gM?, the optimal f will be different in general. 



5 One-Dimensional Cases with Constant Coefficients 

In this section, we make the following assumption 



(5.1) 



ni = n2 = m = 1, 



A2 


Q2 



A{t) 

Q{t) 



B = 





B2 



, C{t) 





C2 



D(-) 





D2 




G2 



, K{-) 



, Rit) = R21 G - 

A^{t) = Au Bi{t)=Bi, Ci{t) = Ci, Di{t)=D,, 
I Qi{t) = Qi, Ri{t) = Ri, Gi{t) = Gi, 

where Ai,Bi,Ci,Di,Qi,Ri,Gi,K are all constants {i = 1,2), and 

(5.2) i?i,J?2>0, Qi,Q2,Gi,G2>0, Ky^O. 
Then the controlled system becomes 

' dXi(t) = [AiXi{t) + Biu{t)]dt + [CiXi{t) + Diu(t)]dW{t), 

(5.3) I dX2{t) = [A2X2{t) + B2U{t)]dt + [C2X2{t) + D2U{t)]dW{t), 

,Xi(0)=ari, X2{t)=KX^{t-Q), 
and the cost functional is 



1 

K 



i e [0,t), 
i e [t,t]. 



(5.4) 



J(xi;«(-),T) = [QiXi{tf +R^u{tf]dt + GiX^{Tf 

+ [Q2X2{tf + R2u{tf]dt + G2X2{Tfy 



Thus, the first component Xi{-) of the state process will be completely terminated from r on. In this 
case, we have 

'0 

P2{t) 



P{t) = 



te [r,T], 



where P2{-) satisfies 
(5.5) 



P2{t) + {2A2 + Cl)P2{t) +Q2- =0, tG [0, T], 



R2+DlP2{t) 



^P2(T) = G2, R2+DlP2{t)>G, 
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and -Pf (•) satisfies 



Ri + DjP[{t) 



(5.6) 



If we denote 



Pl{t) + {2Ai + Ct)Pl{t) + Qi 



F,{P) = {2A + Cf)P 
Then (|5.5p and (|5.6p can be written as 



R^ + DfP 



= 0, te[o,r], 



i = 1,2. 



(5.7) 

and 

(5.8) 



P2(i)+i^2(/'2(t)) =0, te[0,T], 

P2(T) - G2, 



Pr(i)+i^i(Pr(i)) -0, te[0,r], 
Pn0=i^'^2(r)+Gi. 



Note that since ni — 1, the optimal time r of Problem (DOT) is independent of the initial state xi. 
Thus, the optimal time f satisfies 



(5.9) 



The following gives a necessary condition for r. 

Proposition 5.2. Let (|5.ip - (|5.2p hold. Then the optimal time f to Problem (DOT) satisfies tiie 
following condition: 



(5.10) 



>0, 

Fi(i^2p^(f) +Gi) - K^F2{P2if)) { = 0, 

<0, 



f = 0, 

(o,r). 



Proof. We first claim that 



n''(t) = ^P:(t), te[o,r], 



exists for any r G (0, T) and ![''(•) solves 
(5.11) 

To see this, let r S (0, T) and e > small so that r ± e e {0,T). Consider the following: for any t S [0, r], 



n^{t) + Fi{pat))Wit) = o, te [o,r], 

n'^(r) = Pi(i^2p2(r) + Gi) - X2i;^2(P2(r-)). 



P[+^(t) - P[{t) = P2{r + e)- P2{r) 



F^{Pl+'{s))ds 



F,{Pl+'{s))~F,{Pl{s))\ds. 

Then by a standard argument, we have the existence of the following limit: 

P[+-(t)-Pf(t) 



n;(t)^iim. 



t e [0,r], 
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and 



W\[t)^K^P^{r)+F,{Pl{r))+ F'(P[(s))n;(s)ds 

- -K^F2{P2ir)) + Fi {K^P2ir) + Gi) + F' {Pl{s))Ti\{s)ds, t S [0, ^ 



F^{Pl{s))ds 
ds. 



On the other hand, 

Pr^it) - P[{t) = \P2ir - e) - P2M 

^ [f,{P[-^s)) - F.iP^s)) 
Again, by a standard argument, we have the existence of the following limit: 

and 



W_{t) = K^P2{r) + F,{P[{r)) + J Fi{P^ {s))nL{s)ds 

- -K^F2{P2{r))+F^{K^P2{f) + Gi)+ j^^ F[{Pl{s))ir_{s)ds. 
Thus, r i-> -Pf (t) is difFerentiable with derivative 



-Pl{t)^W{t) = w^{t), te[o,r]. 



and n(-) satisfies (|5.1ip . Clearly, 



W{t) = e 



Fi{K^P2{r)+Gi) - K^F2{P2{r)) , t G [0,r] 



Now, since r 1— > Pf(0) attains a minimum at r G (0,r), we have 

n*'(0) = 0, 

which leads to our conclusion for f G (0, T). Now, if f = 0, then 



Finally, if r = T, then 



This completes the proof. 



nO(0) = lim»l^^>0. 
£4.0 e 



n^(o)^iim^^^(°)-^^"(Q)<o. 

£4,0 —e 



If the optimal time r is either or T, our Problem (MLQ) becomes less interesting. Therefore, the 
optimal time f is said to be non-trivial if f G (0,r). From the above, one has the following corollary. 



(5.12) 



Corollary 5.2. Let ^^-^^ hold. Then 

Fi{K^G2 + Gi) - K^F2{G2) > =^ f < T, 
Fi{K^P2iO) + Gi)-K^F2{P2{0)) <0 ^ r>0. 
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Therefore, r is non-trivial if that 

(5.13) Fi{K^G2 + Gi) - K^F2{G2) > > F^iK^P^iQ) + d) - if'F2(^'2(0)). 



Note that in principle, conditions in (|5.13p are checkable. Let us now look at some special cases for 
which we can say something about the optimal time f. Let 

(5.14) D2 = Gi = Q, R2^K = l, B2 ^ 0. 

In this case, we observe the following: 

Fi{K^P + Gi) - K^F2[P) = Fi{P) - F2[P) 



{2Ai + Cf)P + Qi 



{Bi + CiDifP^ 



Ri + D(P 



2{Ai - A2) + Cl - Cl 



P + Qi-Q2 + BiP^ 



{2A2 + Ci)P + Q2- BjP^ 

{Bi+CiDifP'^ _ e(P) 



Ri+DfP Ri + DfP' 



where 



e(P) = DlB^P^ + [(2{Ai - A2) + Cl - ci^Dl + RiBl - {Bi + CiDif 
+ [(2(Ai -A2) + Cl- G|) i?i + (Qi - Q2)D'i\ P + (Qi - Q2)Ri 
Then ([STTO)) is implied by 

(5.15) e(G2) > > e(P2(0)). 

On the other hand, in the current case, the Riccati equation for P2(0 becomes 



(5.16) 
Denote 



P2{t) + {2A2 + Cl)P2{t) + Q2- BlP2{tf = 0, te [0, T], 

P2{T) = G2. 



A+ = 



2A2 + G| ± ^/(2^2 + G|)2 + 4B|Q2 



2Bl 



Then we may rewrite the above Riccati equation as 
(5.17) 



P2{t) - Bl[P2{t) - \+][P2{t) - A_] = 0, i e [o,r], 
P2(r) = G2. 



If 



G2 = A±, 

then the unique solution P2(') is given by 

P2{t) = G2, tG[0,T], 
for which, (|5.15p cannot be true. Therefore, in what follows, we assume that 
(5.18) G2 7^A±. 

Then the solution P2(') is given by 

A+(G2 - \_)eBl(^+-^-){T-t) _ A_(G2 - A+) 



P2{t) - 



(G2 - A_)e^2(^+-^-)(^-*) - (G2 - A+) 



t e [o,r]. 
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We claim that 

^ (G2-A_)e^i(^+-^-)^-(G2-A+) ^ 
In fact, it is easy to see that there is no to e (0, T) such that 

B^[P2{to) - A+][P2(<o) - A-] = P2(to) = 0. 
Otherwise, by the uniqueness of solutions to ODEs, we must have 



X+, te[0,T], ifP2(<o)-A+ = 0, 
A_, te[0,T], if P2(to) -A_ =0, 



both of which contradict (|5.18p . Actually, when (|5.18p holds, by observing the sign of P2{-), one has the 
following: 



(5.19) P2(0) 
In particular, if Q2 ~ 0, then 



>G2, if G2 e (A_,A+), 
<G2, if G2 ^ [A_,A+]. 



^ _ 2A2 + G| 

A+ - ^2 : 

^2 



Consequently, 

G2(2A2 + C|)e(2^^+C2)r J >G'2, 2A2 + G2' > G2B2', 



^^2(0) 



G,5|e(— J)- + 2A2 + CI - G2PI , ^ ^ ^2 < ^^^2. 



Further, let Di = and i?i 1. Then 

e(P) = (p2 _ 52)p2 ^ j^2Ai + G^ - (2^2 + G2))p + Qi. 

Let 

Then 0(-) has a unique positive root: 

_ {lAr + G?) - (2^2 + G|) + ;/ [(2Ai + G2) - (2^2 + G|)] ' + 4(5? _ bDQ^ 



Hence, if 

2A2 + G| > G2PI, 

and 

< G2 < P+ < P2(0), 

then (j5.15p holds and f is non-trivial. 

It is clear that many other cases for which f is non-trivial can be discussed in the similar fashion. 
However, we prefer to omit the details here. 
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